Comparing output results from cellassign

In this notebook, we observe the annotation results obtained from cellassign https://github.com/Irrationone/cellassign Those results were obtained in R in the R markdown named: cellassign.Rmd/cellassign.html

In this exercice we first predict labels for the PBMC3k datasets using:

PMBC3k predition by cellassign is compared with the sig-annot and auto-annot prediction.

PBMC prediction

We load the dataset predicted with Auto-Annot

In this pbm3k data are stored multiples scores and information. Leiden is the leiden clustering on the whole datasets post classical filtering. dblabel is the results of the sig-annot procedure on the whole dataset also after reclustering around T-cells/NK-cells for a finer grain annotation. Finally auto_annot is the results of the auto-annot procedure using Granja and Kotliarov datasets as training sets.

Cellassign prediction

We load the prediction obtained with cellassign and upload the said results in our h5ad object.

cellassign predictions were obtained in R, please see cellassign.html / cellassign.Rmd

Labels to dblabel

We need to map back the labels to dblabels using the dblabel nomenclature (described in besca in CellTypes_v1.tsv) when using the singlecell datasets provided by cellassign (in the celldex package)

Comparing predictions

We fix the palette to keep the same colors for the UMAPs celltypes

Predictions reports generation

bescaF_label vs Sig-annot

bescaFv2_label vs Sig-annot

bescaVvs_label vs Sig-annot

cellassign_label vs Sig-annot

Compare F1 Score

Besca report generate overall models. We retrieved the overall F1 and accuracy score for all reports generated in order to compare those values.

A large number of cells remain unassigned. In particular T cell assignment seems problematic.

END